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Executive  Summary 


The  US  Army  Research  Laboratory  (ARL)  Data  to  Decisions  team  participated  in 
the  Naval  Postgraduate  School-sponsored  Joint  Interagency  Field  Experimentation 
(JIFX)  15-2,  9-13  February  2015,  at  Alameda  Island,  California.  The  ARL  team, 
composed  of  government  and  contractor  scientists,  was  joined  by  teams  of 
engineers  and  computer  scientists  from  the  US  Air  Force  Research  Laboratory 
Wide-Area  Motion  hnagery  (WAMI)  directorate  and  the  Michigan  Institute  of 
Technology-Lincoln  Laboratory  to  demonstrate  how  the  use  of  text  analytics  and 
WAMI  tools  could  be  used  to  support  operational  planning  and  execution. 

The  JIFX  setting  offers  government,  private  sector,  academic,  and  nonprofit 
technologists  an  opportunity  to  engage  directly  with  the  Department  of  Defense, 
US  federal  civilian  agencies,  first  responders,  international  partner  nations,  and 
companies  in  a  semistructured  learning  environment  that  promotes  collaboration 
and  innovation  while  developing  communities  of  interest  and  knowledge-sharing 
activities. 

The  JIFX  environment  is  a  unique  opportunity  for  developers  to  engage  with  end- 
users  in  new  or  emerging  concepts  and  solutions.  By  interacting  directly  with  end 
users  in  a  replicated  operational  environment,  technologists  are  able  to  leverage  the 
knowledge  gained  to  improve  or  enhance  their  products,  steer  development  to 
requirements,  and  reduce  the  time  required  to  transition  a  technology,  often  saving 
in  development  time  and  dollars. 
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1.  Introduction 


The  Data-to-Decisions  (D2D)  research  program  in  the  area  of  text  analytics  is  an 
Office  of  the  Secretary  of  Defense  (OSD),  Office  of  the  Assistant  Secretary  of 
Defense,  Research  and  Engineering  (ASD  [R&E])  multiyear,  funded,  and  directed 
program  that  is  competitively  awarded.  The  D2D  program  provides  an  opportunity 
to  conduct  objective  assessments  of  analytics  for  rapid  maturation  to  support 
understanding  and  develop  improvements  in  mission  effectiveness  in  time- 
constrained  and  data-intensive  environments. 

Military  decision  making  requires  supporting  data  from  a  variety  of  sources  and  in 
a  wide  range  of  formats  to  achieve  success  in  its  many  missions.  Text  and  language 
play  an  important  part  in  many  military  missions  by  providing  the  key  data  required 
for  understanding  cultures,  attitudes,  events,  and  relationships  that  serve  as  the  basis 
of  many  missions  conducted  around  the  world.  With  advances  and  increased  use  of 
Internet  and  mobile  communications,  text  infonnation  is  available  in  unprecedented 
amounts  and  fonnats,  presenting  a  unique  opportunity  to  gain  understanding 
through  text  analytics.  In  the  context  of  the  D2D  research  program,  text  analytics 
refers  to  identification  of  a  set  of  linguistic,  statistical,  and  machine  learning 
techniques  that  model  and  structure  the  information  content  of  textual  sources  for 
exploratory  data  analysis,  research,  and  investigation.  Text  analytics  involves 
information  retrieval,  lexical  analysis  to  study  word  frequency,  data-mining 
techniques  including  link  and  association  analysis,  visualization,  and  predictive 
analytics. 

The  Department  of  Defense  (DOD)  recognizes  the  potential  for  text  analytics  to 
play  a  vital  role  in  future  capabilities  that  inform  timely  and  accurate  situational 
awareness  in  time-constrained,  uncertain,  and  complex  environments  by 
developing  capability  in  contextual  understanding,  event  prediction,  and  machine 
translation  and  processing.  The  D2D  program  expects  to  achieve  these  capabilities 
through  a  spiral  development  process  that  begins  with  a  scenario  that  exhibits 
several  text  analytics  challenges  described  in  the  following  section.  This  scenario 
supports  an  end-to-end  demonstration  prototype  for  each  spiral  to  1)  analyze 
performance  to  identify  critical  science  and  technology  needs,  2)  provide  data  that 
can  be  distributed  in  support  of  science  and  technology  development,  3)  provide 
demonstrations  of  key  technologies  enabling  assessment  of  technology  maturity,  4) 
provide  system  solutions,  system  improvements,  and  alternative  concepts  of 
operations,  and  5)  develop  a  predictive  capability  for  operations. 
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2.  Wide-Area  Motion  Imagery 


In  recent  years,  the  military  has  fielded  several  wide-area  motion  imagery  (WAMI) 
sensor  systems  (e.g.,  Constant  Hawk  and  Angel  Fire)  and  is  in  the  process  of 
acquiring  newer,  more  capable  systems.  These  systems  persistently  monitor  fixed 
geographic  locations  for  long  periods  of  time  using  electro-optic  sensors.  Some 
store  their  WAMI  data  onboard  and  download  it  at  the  end  of  each  mission  for 
postevent  analysis  and  exploitation.  Others  provide  operational  support  through 
real-time  transfer  of  the  WAMI  data.  Archived  WAMI  data  are  used  mainly  for 
postevent  analysis  or  to  perfonn  network  analysis  of  a  facility  of  interest  (as 
identified  through  other  intelligence  sources).  In  either  case,  analysts  try  to 
determine  all  the  entities  going  to  and  from  an  event  or  facility  of  interest  to 
ascertain  the  unique  sources  or  destinations  of  those  entities  as  a  means  to 
understand  their  participation  in  an  event  or  their  relationship  in  some  network.  Due 
to  the  manual  techniques  used,  these  analyses  typically  take  many  hours  to  many 
days  to  complete,  and  the  end  products  are  text  reports  with  simple  graphics.  These 
reports  are  not  machine  readable  (analysts  produce  shapefiles  for  only  a  tiny 
fraction  of  reports)  and  therefore  preclude  additional  automated  analysis  and 
exploitation.  Typically,  real-time  WAMI  is  used  to  give  the  battle  commander 
better  situational  awareness.  However,  only  the  most  simplistic  use  of  the  imagery 
is  possible  because  of  the  time-consuming  and  manual  nature  of  the  available 
analysis  tools. 

As  part  of  the  overall  persistent  surveillance  mission,  the  military  services  are 
expanding  the  deployment  and  development  of  airborne  WAMI  sensors.  These 
sensors  operate  in  predominantly  urban  settings,  are  capable  of  constantly 
monitoring  many  square  kilometers  for  many  hours,  and  generate  terabytes  of  data 
per  mission.  Operationally,  WAMI  data  are  exploited  for  events  of  interest,  both 
forensically  and  in  real  time.  The  real-time  efforts,  like  Angel  Fire,  are  limited  to  a 
small  number  of  subframes  and  used  primarily  for  force  protection.  Events  of 
interest  can  include  starting  points  and  destinations  of  tracks  and  nodes  for  related 
entities  within  the  persistent  field  of  view.  They  can  also  include  activity  and  event- 
based  normalcy  and  anomaly  detections  such  as  unique  driving  behaviors  occurring 
before  the  detonation  of  a  suicidal  vehicle.  Other  types  of  events  can  be  used  to 
discover  or  highlight  “patterns-of-life”  associated  with  a  variety  of  network  types, 
including  social,  political,  regional,  and  economic  or  military  networks. 

The  challenge  is  to  identify  potential  threats  based  on  the  accumulation  and 
correlation  of  multiple  events  and  anomalies,  and  issue  timely  alerts  with  a  minimal 
number  of  false  alarms.  WAMI  data  are  having  an  increasing  impact  on  operational 
outcomes,  but  current  efforts  to  exploit  these  data  are  mostly  manual  and  require 
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hours  to  days  of  painstaking  analysis  to  produce  results.  The  tedious  nature  of 
current  exploitation  capabilities  limits  the  ability  to  fully  utilize  the  available  data. 
Consequently,  critical  battlefield  questions  go  unanswered  and  timely  threat  cues 
are  missed. 

The  primary  information  elements  in  WAMI  data  are  entities  in  the  context  of 
roads,  buildings,  and  other  scene  features.  Exploitation  of  these  entities  yields 
tracks  but  in  a  complex  urban  environment,  these  tracks  are  severely  fragmented 
due  to  occlusions,  stops,  and  other  factors.  Within  these  localized  events, 
algorithms  that  discover  relationships  and  anomalies  that  are  indicative  of 
suspicious  behavior,  match  previously  learned  threat  activity,  or  match  user- 
defined  threat  activity  should  also  be  incorporated.  While  the  localized  events  may 
each  occur  over  a  small  spatiotemporal  window,  the  overall  threat  activity  sequence 
may  span  a  much  larger  spatiotemporal  window. 

3.  Joint  Interagency  Field  Experiment  Program 

The  Joint  Interagency  Field  Experimentation  Program  (JIFX)  began  in  2012  under 
the  sponsorship  of  the  OSD  and  the  Department  of  Homeland  (DHS)  security.  The 
JIFX  setting  offers  government,  private  sector,  academic,  and  nonprofit 
technologists  an  opportunity  to  engage  directly  with  DHS,  DOD,  federal  civilian 
agencies,  and  first  responders  in  a  semistructured  learning  environment  promoting 
collaboration  and  innovation  while  fusing  interactive  community  networking  and 
knowledge-sharing  activities. 

The  JIFX  environment  is  a  unique  opportunity  for  developers  to  engage  with  end 
users  in  new  or  emerging  concepts  and  solutions.  By  interacting  directly  with  end 
users  in  a  replicated  operational  environment,  technologists  are  able  to  leverage  the 
knowledge  gained  to  improve  or  enhance  their  products,  steer  development  to 
requirements,  and  reduce  the  time  required  to  transition  a  technology,  often  saving 
millions  of  dollars  in  research  and  development. 

Each  quarter,  a  JIFX  event  uses  different  methods  of  interaction,  all  of  which  focus 
on  end  user  input.  This  reflects  and  addresses  the  most  complex  challenges 
identified  by  those  directly  engaged  in  homeland  defense  and  security. 

Approximately  90  days  before  each  JIFX  event,  the  OSD  and  the  Naval 
Postgraduate  School  (NPS)  publish  a  Request  for  Information  (RFI).  The  RFI  seeks 
concept  papers  addressing  specific  technology  challenges. 
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Select  respondents  are  subsequently  invited  to  participate  in  the  JIFX  experiments. 
The  selection  of  respondents  is  based  on  the  relationship  of  the  technology  to  a 
particular  class  or  level  of  capability  that  can  be  provided  to  the  mission-sets  in  the 
RFI.  Other  considerations  include  technical  maturity,  relevance  of  or  adaptability 
to  missions,  and  relevance  to  current  operational  needs. 

The  JIFX  success  is  enhancing  the  discovery  of  technologies  and  operation  and 
performance  capabilities. 

Major  objectives  include  the  following: 

•  Explore  the  viability  of  new  concepts  as  solutions  for  identified  current  and 
future  capabilities 

•  Provide  a  venue  for  short-fused  experimentation  requirements 

•  Explore  concepts  of  operation  and  human  systems  integration 

•  Explore  technologies  and  solutions  in  support  of  whole-of-govemment, 
humanitarian  and  disaster  relief,  critical  infrastructure/key  resources 
protection;  and  interagency  communications 

4.  OODA  Loop  as  a  Framework 

The  phrase  OODA  loop  refers  to  the  decision  cycle  observe,  orient,  decide,  and  act 
(OODA)  developed  by  Major  John  Boyd,  US  Air  Force.  Boyd  developed  the 
concept  to  explain  how  to  direct  one’s  energies  to  defeat  an  adversary  and  survive. 
Boyd  emphasized  that  “the  loop”  is  actually  a  set  of  interacting  loops  that  are  to  be 
kept  in  continuous  operation  during  combat.  He  also  indicated  that  the  phase  of  the 
battle  has  an  important  bearing  on  the  ideal  allocation  of  one’s  energies.1 

For  JIFX,  the  team  used  the  OODA  loop  construct  (Fig.  1)  to  arrange  and 
characterize  how  the  various  technology  components  might  interact  with  each  other 
and  support  the  flow  of  information. 
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Observe  Orient  Decide  Act* 


Fig.  1  Boyd  OODA  Loop  (Adapted  from  Boyd  JR.  The  essence  of  winning  and  losing. 
Unpublished  briefing  slides;  1996  Jan.) 

The  majority  of  the  text  analytic  tools  provided  by  the  team  focused  on  the  observe 
and  orient  portions  of  the  OODA  loop  model.  Our  interpretation  of  the  observe 
phase  was  ingesting  large  data  sets  and  processing  the  data  using  various 
tools/algorithms  to  provide  some  orientation  on  where  issues  might  be  occurring  in 
a  region  of  interest.  The  processed  data  resulted  in  a  variety  of  document 
summaries,  sentiment  analysis  of  various  groups,  relationship  analysis  of  the 
various  participants  and  groups  identified  in  the  data,  discourse  analysis  of  what 
people  and  groups  were  saying,  and  finally,  with  the  use  of  imagery  analysis, 
forensic  image  analysis  of  specific  areas  was  possible.  The  tools  and  their 
implementation  into  to  a  workflow  are  discussed  in  the  next  section. 

5.  Concept  Demonstration 

The  participation  of  the  combined  government,  industry  and  academia  team  led  by 
the  US  Anny  Research  Laboratory  (ARL)  was  to  demonstrate  the  concept  of  using 
text  analytics  to  queue  a  sensor  and  vice  versa.  Our  process  (Fig.  2)  started  with  a 
predictive  analysis  of  the  country  of  interest.  In  this  case,  the  team  focused  on 
Nigeria  and  used  open  source  data  collection  methods  for  compiling  the  necessary 
data  for  analysis.  A  workflow  was  developed  by  the  team  to  demonstrate  a  cohesive 
flow  of  information  and  data  products  across  the  assembled  technology  capabilities. 
Figure  2  shows  the  capability  spectrum  mapped  to  the  OODA  loop.  In  green  blocks 
are  the  text  analytics  performers  with  a  high-level  description  of  each  capability.  In 
blue  blocks  are  the  WAMI  workstation  and  sensor  inputs.  In  brown  blocks  are  the 
scenario  vignettes  that  were  used  to  drive  the  information  processing.  Finally,  in 
dark  brown  blocks,  are  the  time  lines  used  in  the  scenario.  These  time  lines  replicate 
actual  information  processing  in  military  operations,  with  a  combination  of 
historical  and  real-time  data  inputs  for  the  decision-making  process. 
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Unfolding 

Circumstances 


SAE  -  Early  Warning  System, 
Discourse  &  Sentiment  Analysis 


IAI  -  Relationship  Analysis 


BGU  -  Document  Summary 


UW  -  Document  Summary  & 
Relationship  Analysis 


KBSI  -  Relationship  Analysis 

BBN  -  Relationship  Analysis 
ACS  -  Relationship  Analysis 


WAMI  Analysts  Workstation 


Boko  Haram  Attack  Concept 
Developmentand  Financing 


Target  Identification  and  Surveillance 
Recruit,  Train  and  IED  Preparation 


WAMI  Sensor 


Large  Corpus  (12  +  Months  of  Data)  ■  Monthly  /  Weekly  /  Daily  Data  |  Near  Real  time  Data  ||  Real  time  Data 


Fig.  2  Multisource  data  analytics  framework 


6.  Scenario 


For  our  team’s  participation  in  JIFX,  we  constructed  a  scenario  focused  on 
activities,  places,  groups,  and  people  in  Nigeria.  The  focus  on  Nigeria  enabled  the 
team  to  examine  a  variety  of  different  data  sources  and  groups  of  people.  Because 
of  the  wealth  of  information  and  reports  available  in  the  public  domain,  we  chose 
the  terrorist  group  Boko  Haram  as  the  focus  for  our  analysis  effort.  As  part  of  the 
scenario,  we  developed  a  thread  for  an  improvised  explosive  device  attack  against 
a  foreign  official.  Our  group  objectives  included  the  following: 

•  Predict  the  event,  location,  time 

•  Conduct  sentiment  analysis 

•  Conduct  discourse  analysis 

•  Identify  potential  victim 

•  Identify  specific  perpetrators 

•  Demonstrate  the  Text  Analytics/WAMI  provider  capabilities  and  the 
complimentary  nature  of  each  tool 

•  Final  goal:  to  provide  actionable  intelligence  to  a  WAMI  sensor  operator 
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The  scenario  data  set  that  was  developed  for  this  experiment  included  more  than 
40,000  documents  from  5  different  sources.  The  “ground  truth”  data  set  included 
more  than  700  different  tactical  reports,  intelligence  reports,  police  reports, 
speeches,  tweets,  and  various  other  documents.  The  ground  truth  data  set  identified 
relationships  among  groups  of  people  and  identified  specific  people  of  interest, 
planned  meetings,  and  events.  The  ground  truth  was  correlated  to  the  open  source 
data  sets  that  were  developed  from  reports  on  AllAfrica.com,  and  newspaper 
articles  from  Nigeria  Guardian  News,  Tribune  News,  and  PM  News  Nigeria.  Figure 
3  presents  a  graphic  description  of  the  data  set  contents.  Each  of  the  participating 
text  analytics  technologies  used  the  data  to  provide  insights  into  the  planned 
terrorist  activities,  the  various  social  groups,  and  their  roles.  In  this  sense,  the 
scenario  and  data  set  provided  an  element  of  cohesion  and  integration  across  the 
individual  technologies. 

JIFX  15-2  Datasets  ARL 


Nigeria  Guardian 
News,  4005 


Ground  Truth, 
722 


.  JribuneNews, 
'  ^  573 


PM  News 
Nigeria,  17472 


AIIAfrica.com, 

19038 


July  2013 -Nov  2014 
Total  Documents  =41,810 
Key  Elements 

o  Black  rrarket  -  1,660  artces 
o  Drug  srruggng  -  417  artCes 
o  zc<c  Haram  -  17,696  artces 
o  Ansa-u  -  193  art  ces 
o  Piracy  -  1,493  artCes 
o  Abubake*  Shekau  -  76  artces 
o  Momodu  Barra  -  60  artces 
o  Kabiru  Sokoto  -  76  artces 
o  Gu  XiaojTe  -  13  artces 

Ground  Truth 

o  Jlhadlst  publication  -  1 
o  Speeches,  tweets,  bogs,  and  various 
othe-*  documents  -  86 
o  nte  gence  nfonraton  Repots  (HR) 
-105 

o  Police  Repots  -  60 
o  actca  Repots  (TACREP)  -  470 
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Fig.  3  Data  set  contents 


7.  Participant  Experiment  Summaries 


This  section  includes  a  brief  summary  of  the  experiments  and  demonstrations  that 
the  joint  ARL/US  Air  Force  Research  Laboratory  (AFRL)  team  presented  during 
JIFX  15-2. 
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7.1  US  Army  Research  Laboratory 


Topic:  Multi-Source  Data  Analytics 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  The  Multi-source  Data  Analytics  program  demonstrated  how  imagery 
and  text  analytics  can  be  used  together  for  improved  threat  detection,  contextual 
understanding,  event  prediction,  and  decision  making.  Advances  in  WAMI 
technology  applications  were  recently  demonstrated  in  Trident  Spectre  14.  These 
applications,  in  addition  to  technology  capabilities  developed  in  the  text  analytics 
program,  were  demonstrated  for  cross-cueing  of  platforms.  A  portion  of  a  scenario 
developed  for  the  WAMI  demonstration  was  expanded  for  analysis  of  text  for 
content  understanding  and  event  prediction.  These  analytics  provide  tools  that 
reduce  cognitive  workload  of  intelligence  analysts  seeking  knowledge  in  very  large 
document  sets.  Sample  technologies  included  summarization,  social  network 
extraction,  and  foreign  language  exploitation  of  relations  in  networks,  discourse 
and  sentiment  analysis,  and  predictions  of  regional  conflict.  Cross-cueing  events 
included  areas  suggested  for  WAMI  watch-boxes,  potential  individual/groups  for 
monitoring,  and  contextual  understanding  behind  observed  WAMI  activities. 

7.2  Strategic  Analysis  Enterprises,  Inc. 

Topic:  Forecasting  Conflict  &  Estimating  Effects  of  Courses  of  Action  Using  Near- 
Real  Time  Social  Media  and  Group-Level  Discourse  Data  and  Models 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  Strategic  Analysis  Enterprises,  Inc.  (SAE),  extracted  sentiment, 
emotions,  events,  and  discourse  markers  from  near  real-time  social  media  and  news 
feeds.  SAE  displayed  the  trends  over  time  and  space  and  used  the  data  in  forecasting 
models  to  forecast  destabilizing  events  across  time  and  space.  SAE  then  analyzed 
different  potential  courses  of  action  using  matched  observation  and  counterfactual 
analysis  to  mitigate  the  occurrence  of  destabilizing  events  and  assess  likely  impacts. 
The  specific  scenario  focused  on  a  port  visit  to  Lagos,  Nigeria,  and  the  likely 
environment  a  commander  would  encounter  upon  arrival.  SAE  provided  6-month 
forecasts  of  hotspots  in  other  regions  of  Nigeria  focused  on  Boko  Haram,  an  Islamic 
insurgent  organization,  and  its  likely  violent  activities  across  various  regions  of 
Nigeria.  SAE  mined  social  media,  news  feeds,  and  Boko  Haram  YouTube  videos 
and  created  quantitative  indicators  of  societal  fear  and  anger  toward  the  government 
and  Boko  Haram.  They  also  analyzed  the  group’s  messages  in  terms  of  cognitive 
complexity  and  idea  density.  Using  these  data,  SAE  forecasted  likely  trajectories 
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of  violent  events  across  Nigeria  6  months  into  the  future.  In  addition  to  the  NPS 
JIFX  demonstration,  the  SAE  tools  and  techniques  have  been  successfully  applied 
to  a  STRATCOM  program  of  record.  SAE  is  also  working  with  AFRL  to  transition 
this  technology  to  the  National  Air  and  Space  Intelligence  Center  for  operational 
use.  The  SAE  forecasts  are  used  by  the  Central  Intelligence  Agency,  Department 
of  State,  and  many  combatant  commands  through  the  Lockheed-Martin  Worldwide 
Integrated  Crisis  Early  Warning  System,  in  which  SAE  analytics  is  a  critical 
component. 

7.3  Charles  River  Analytics,  Inc. 

Topic:  Collaborative  Narratives  for  Social  Media  Information  Fusion 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  The  wealth  of  data  provided  by  modem  social  media  information  fusion 
tools  exceeds  the  ability  of  intelligence  analysts  to  effectively  process,  exploit,  and 
disseminate  actionable  intelligence.  This  issue  arises  because  these  information 
fusion  tools  define  information  value  and  usage  statically,  and  consider  analysts 
solely  as  resource-unbounded,  detenninistic  reasoning  mechanisms,  rather  than 
dynamic,  critically  thinking  individuals  who  may  exhibit  biases  and  cognitive 
overload.  To  overcome  those  issues,  Charles  River  Analytics,  Inc.  (CRA), 
developed  a  system  for  using  Knowledge  of  Narratives  for  Optimizing  Workspaces 
to  Enhance  Reasoning  (KNOWER).  KNOWER  provides  a  revolutionary  approach 
to  infonnation  fusion  that  uses  narratives  to  enable  analytical  capabilities  that 
support  analysts  in  exploiting  modern  data  collection  capabilities.  Narratives  are  a 
natural  and  prevalent  fonn  of  communication  used  to  relate,  summarize,  and 
emphasize  infonnation  to  make  sense  of  the  world.  Narratives  provide  temporal, 
spatial,  and  intentional  infonnation  about  characters  and  their  actions  in  an  easily 
understandable  fonnat.  KNOWER  harnesses  existing  and  novel  fusion  techniques 
with  respect  to  framing  concepts  in  terms  of  nanatives  in  supporting  the 
intelligence  analyst’s  investigative  process.  Additionally,  KNOWER  integrates  the 
CRA  Collaborative  Visual  Exploitation  and  Reasoning  Tool  to  provide 
semantically  rich,  chat-based  collaboration  mechanisms  that  supports  higher-level 
infonnation  sharing  of  concepts,  entities,  relationships,  narratives,  and  products. 
The  transition  plan  for  the  KNOWER  software  is  the  Army’s  Distributed  Common 
Ground  Station  for  Special  Operations  Forces’  use. 
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7.4  Knowledge  Based  Systems,  Inc. 


Topic:  Text  Analytics  Situational  Awareness  Toolkit  (T-SAT) 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  T-SAT  provides  a  comprehensive  text  analytics-based  enabling 
capability  for  improved  threat  detection,  contextual  understanding,  event 
prediction,  and  decision  making.  T-SAT  uses  ontology-driven  text  processing 
methods  to  extract  infonnation  and  knowledge  from  large  amounts  of  multisource 
text  data  to  recognize  complex  relationships  contained  within  the  information  and 
presents  the  results  in  a  concise  and  easy-to-comprehend  way.  By  semantically 
tagging  the  ingested  data,  T-SAT  organizes  and  manages  information  to  enable 
rapid  and  more  meaningful  responses  to  queries  and  customizable  report 
generation.  Originally  designed  for  the  intelligence  community,  the  use  of  T-SAT 
in  this  experiment  demonstrated  the  ability  to  analyze  different  types  of  data  such 
as  news  articles,  blogs,  Twitter,  and  simulated  intelligence  message  feeds.  T-SAT 
provided  directed  insight  into  coordinated  activity  and  relationships  among  people, 
organizations,  and  locations  (social  and  geospatial  networks)  as  well  as  perceived 
threat  events  or  behaviors  (event  networks).  With  the  ability  to  recognize  complex 
events  and  social  networks,  the  T-SAT  application  provides  a  critical  enabling 
capability  to  address  the  dynamically  changing  demands  of  intelligence  vigilance. 

7.5  Ben-Gurion  University  of  the  Negev 

Topic:  Enhanced  Document  Summarization 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  MUSEEC  (Multilingual  SEntence  Extraction  and  Compression) 
application  is  a  part  of  a  program  aimed  at  identifying  the  salient  parts  of  a  single 
text  document  or  a  collection  of  topic-related  text  documents  in  multiple  languages 
and  presenting  the  extracted  text  summary  to  the  user  in  a  human-readable  and 
coherent  form.  MUSEEC  provided  automatically  generated  summaries  for  the 
documents  provided  in  JIFX  15-2  data  with  the  named  entities  (NEs)  recognized 
and  marked.  The  summaries  and  their  geo-tagging  infonnation  (NEs  representing 
locations)  were  used  by  a  WAMI  engine  to  relate  described  events  to  a  particular 
geo-location.  The  MUSEEC  user  interface  allows  the  size  of  the  produced 
summaries  to  be  controlled  in  terms  of  the  number  of  characters,  words,  and 
sentences  or  a  fraction  of  the  original  document. 
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7.6  Intelligent  Automation,  Inc. 


Topic:  Search,  Analyze,  and  Visualize  Social  Media 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  Intelligent  Automation,  Inc.  (IAI),  collected  real-time  social  media  data 
(e.g.,  Twitter)  and  provided  the  capability  to  search,  analyze  and  visualize  large 
collections  of  Twitter  data  via  text  and  graph  analytics  using  IAI’s  cloud -based 
software  Scraawl.  In  particular,  IAI  presented  1)  basic  statistics  of  these  data  (e.g., 
top  users,  hashtags,  mentions,  and  URLs),  2)  advanced  analytics  (e.g.,  community 
detection,  influential  users,  sentiment  analysis,  and  topic  modeling),  3) 
visualization  (e.g.,  timelines,  heat  maps,  influence  network  maps,  and  community 
clustering),  and  4)  advanced  search  (e.g.,  keyword  search,  time  and  location 
filtering,  and  exclusion  and  inclusion  of  searches).  The  output  was  an  intelligence 
report  summarizing  the  key  actors,  influencers,  communities,  organizations,  and 
the  relationships  among  the  aforementioned  entities  and  events  of  interest. 

7.7  Air  Force  Research  Laboratory/RYA  (Layered  Sensing 
Exploitation  Division) 

Topic:  Multi-Source  Data  Analytics  using  Wide  Area  Motion  Imagery  (WAMI) 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  Advances  in  WAMI  technology  applications  were  recently 
demonstrated  in  Trident  Spectre  14  including  near  real-time  forensic  vehicle  and 
dismount  detection  and  tracking,  activity  (stops,  starts,  turns,  aimless  driving,  etc.) 
detection,  and  heat  maps  of  tracks  and  activities.  These  applications,  in  addition  to 
technology  capabilities  developed  in  the  text  analytics  program  (e.g.,  document 
summarization,  social  network  extraction,  foreign  language  exploitation  of 
relations  in  networks,  discourse  and  sentiment  analysis,  and  predictions  of  regional 
conflict),  were  demonstrated  for  cross-cueing  of  platfonns.  Cross-cueing  events 
included  areas  suggested  for  WAMI  watch-boxes,  potential  individual/groups  for 
monitoring,  and  contextual  understanding  behind  observed  WAMI  activities. 
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7.8  Applied  Communication  Sciences,  Inc. 


Topic:  Dynamic  Event  Extraction  and  Prediction  (DEEP) 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  The  goal  of  the  program  was  to  develop  technologies  that  enable 
intelligence  analysts  to  rapidly  extract  infonnation  about  events,  entities,  and 
relations  from  large  volumes  of  text  using  cutting-edge  information  extraction 
technologies.  This  enabled  analysts  to  rapidly  focus  on  the  current  interest  of 
terrorists  and  their  use  of  code  words  to  “hide”  infonnation.  It  also  enabled  the 
analyst  to  identify  persons  and  events  that  are  related  to  each  other. 

7.9  Kitware,  Inc. 

Topic:  Waterfall 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  WATERFALL  is  a  software  services  solution  that  leverages  a 
combination  of  commercial  off-the-shelf  (Knowmadics  CASES  and  SilverEye)  and 
government  off-the-shelf  (GOTS)  (ISRFabric)  software  applications  to  enable  the 
tactical  operator  to  purchase  a  4G  handset  on  the  local  economy  and  begin  using  it 
as  a  location-based  collection  asset  with  minimal  user  configuration.  For  this 
experiment,  the  tactical  operators  connected  their  4G  smartphone  to  TrustComm’s 
Ka/4G/LTE  comins  backbone  provided  for  the  event.  The  data  was  transported  via 
the  ISRFabric  (GOTS)  for  display  to  operators  working  across  network  domains. 

7.10  University  of  Washington 

Topic:  Accurate,  Intuitive,  Scalable  Text  Analytics 

JIFX  Area  of  Focus:  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Summary:  University  of  Washington’s  (UW’s)  research  focused  on  2  key  threads: 
1)  joint  entity  linking  and  infonnation  extraction,  and  2)  large-scale  multidocument 
summarization. 

1)  Joint  entity  linking  and  information  extraction.  Motivated  by  the  need  for 
corpus-level  analysis  of  text,  there  has  been  an  increasing  interest  in 
modeling  global  entities  and  relations  that  appear  in  the  text  across  multiple 
documents  and  aligning  them  to  external  knowledge  bases  such  as  Freebase. 
Current  systems  target  entity  linking,  cross-document  coreference,  and 
relation  extraction  as  separate  tasks,  resulting  in  little  or  no  exchange  of 
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information  across  these  problems.  However,  these  tasks  are  inherently 
dependent  on  each  other,  and  models  of  text  that  capture  these  dependencies 
can  result  in  accurate  extraction  of  facts  from  large  text  corpuses.  UW 
focused  on  joint  modeling  of  this  process  toward  the  goal  of  achieving 
significant  gains  in  the  quality  of  infonnation  extraction. 

2)  Large-scale  multidocument  summarization.  The  explosion  in  the  number  of 
documents  currently  available  over  the  Web  necessitates  automated 
approaches  that  organize  and  summarize  large  document  collections  on  a 
complex  topic.  Existing  methods  for  multidocument  summarization 
condense  10-15  documents  into  a  short  flat  summary;  they  are  insufficient 
for  large-scale  summarization.  For  large-scale  summarization,  summarizers 
are  needed  that  organize  the  infonnation  coherently  and  enable  personalized 
interaction  with  the  summary  so  that  users  can  explore  the  various  aspects 
of  infonnation  in  different  levels  of  detail  based  on  individual  interest.  UW 
focused  on  implementing  such  a  system  under  a  novel  paradigm, 
Hierarchical  Summarization. 

7.11  Raytheon  BBN  Technologies,  Inc. 

Topic:  Automatic  Analytics  for  Non-English  Text  with  Bilingual  Tools 
JIFX  Area  of  Focus:  Analytics  of  Non-English  Text 

Summary:  Infonnation  extraction  technologies  transfonn  free  form  prose  (e.g., 
news  articles,  eyewitness  reports,  and  tweets)  into  nonnalized  information  that  can 
be  integrated  into  automatic  analytics  (e.g.,  structured  summaries,  visualizations  of 
social  networks,  geo-location,  and  forecasting  models).  This  can  be  done 
successfully  for  English  text;  however,  traditional  approaches  have  not  perfonned 
accurately  on  non-English  sources.  In  a  disaster  relief  situation  outside  of  the 
United  States,  it  is  likely  that  news,  social  media,  and  other  critical  sources  of 
situational  awareness  are  likely  to  appear  in  the  local  language.  While  tools  exist  to 
automatically  translate  non-English  text,  the  output  of  even  the  best  automatic 
translations  frequently  contains  errors  that  hinder  the  accuracy  of  a  pipeline  to  1) 
translate  text;  2  apply  infonnation  extraction  tools.  Machine  translations’  tendency 
to  make  errors  on  names  (e.g.,  not  translating  Boko  Haram  or  ISIS)  is  a  particular 
challenge  for  the  pipelined  approach  to  infonnation  extraction  from  non-English. 

BBN  has  designed  a  capability  that  operates  over  the  source  language  text  in 
conjunction  with  automatic  translation  to  improve  the  accuracy  of  automatic 
extraction.  As  a  proof  of  concept,  BBN  developed  relation  extraction  tools  to 
extract  relations  (i.e.,  “person-Is-Acquainted-With-Person,”  “person-Is-Member- 
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Of-Organization”,  “person- Visited-Location”,  and  “person-Made-Statement-On- 
Topic”).  These  extractors  were  applied  to  non-English  documents  (in  BBN’s 
prototype  Chinese)  and  the  output  of  the  extractors  was  used  to  produce  concise 
structured  summaries.  The  tool  is  designed  to  require  minimal  manual  input  to 
automate  the  relation  extraction  process  for  a  new  relation  in  a  new  language.  BBN 
achieved  this  goal  by  allowing  initialization  of  a  new  extractor  using  either  non- 
English  or  English  examples  and  by  leveraging  as  much  as  they  could  from  the 
machine  translation  while  using  the  access  to  the  source  text  to  account  for 
translation  errors. 

8.  Conclusion 


Advanced  text  analytics  capabilities  were  demonstrated  in  a  logically  coherent 
workflow  pipeline  that  supported  the  military  OODA  loop  and  allowed  cross- 
cueing  of  a  WAMI  sensor.  The  aggregated  findings  of  these  text  processing 
capabilities  allowed  the  targeted  use  of  the  WAMI  sensor  (albeit  in  a  notional  sense) 
for  rapid  detection  of  mission-critical  events.  The  creation  of  a  very  large  text 
database  from  open  source  data  provides  a  relevant  and  unclassified  foundation  for 
continued  development  of  text  processing  capabilities.  The  ground  truth  features  of 
the  data  set  provide  a  valuable  contribution  to  the  test  and  evaluation  of  emerging 
technologies. 

Future  science  and  technology  goals  for  this  domain  include  closer  applications  of 
text-based  knowledge  and  WAMI  views.  A  near-term  exploration  is  to  develop 
overlays  of  local  context  for  a  geographic  region,  obtained  from  text  analytics,  for 
near  or  real-time  imagery.  Similar  to  how  weather  patterns  are  tracked  on  a  map  for 
local  weather  forecasts,  violence  trends,  social  media  topic  trends,  and  sentiment 
trends  can  be  overlaid  on  a  geographic  map  to  enhance  contextual  understanding  of 
imagery  features. 

The  NPS  JIFX  demonstration  venue  has  been  shown  to  be  valuable  for  emerging 
technologies  at  mid-range  technology  readiness  levels.  The  access  to  Combatant 
Command  science  and  technology  officers  supports  communication  between 
scientists  and  military  advisors,  which  serves  to  enhance  the  military  relevance  of 
new  capabilities  developed  by  university  and  industry  technologists  not  familiar 
with  military  tactics,  techniques,  and  procedures. 
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